W 4 : Who ? When ? Where ? What ? A Real Time System for Detecting and Tracking

W 4 is a real time visual surveillance system for detecting and tracking people and monitoring their activities in an outdoor environment. It operates on monocular grayscale video imagery, or on video imagery from an infrared camera. Unlike many of systems for tracking people, W 4 makes no use of color cues. Instead, W 4 employs a combination of shape analysis and tracking to locate people and their parts (head, hands, feet, torso) and to create models of people's appearance so that they can be tracked through interactions such as occlusions. W 4 is capable of simultaneously tracking multiple people even with occlu-sion. It runs at 20 Hz for 320x240 resolution images on a dual-pentium 200 PC. 1 Overview of the W 4 System W 4 is a real time system for tracking people and their body parts in monochromatic imagery. It constructs dynamic models of people's movements to answer questions about what they are doing, and where and when they act. It constructs appearance models of the people it tracks so that it can track people (who?) through occlusion events in the imagery. In this paper we describe the computational models employed by W 4 to detect and track people and their parts. These models are designed to allow W 4 to determine types of interactions between people and objects, and to overcome the inevitable errors and ambiguities that arise in dynamic image analysis (such as instability in segmen-tation processes over time, splitting of objects due to coincidental alignment of objects parts with similarly colored background regions, etc.) W 4 employs a combination of shape analysis and robust techniques for tracking to detect people, and to locate and track their body parts. It builds \appearance" models of people so that they can be identiied after occlusions or after other interactions during which W 4 cannot track them individually. W 4 has been designed to work with only monochro-matic video sources, either visible or infrared. While most previous work on detection and tracking of people has relied heavily on color cues, W 4 is designed for outdoor surveillance tasks, and particularly for night-time or other low light level situations. In such cases, color will not be available, and people need to be detected and tracked based on weaker appearance and motion cues. W 4 is a real time system. It currently is implemented on a dual processor Pentium PC and …